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To All Whom It May Concern: 

Be It Known That We, Gregory Hobson, Jerry Moore, John R. Wootton, James 
C. Myers, and Jeffrey J. Orth, citizens of the United States, residents of the cities of St. 
Charles, Florissant, St. Louis, Florissant, and Fenton, respectively, each of the State of 
Missouri, whose post office addresses are 17 Upper Dardenne Farms Drive, St. 
Charles, Missouri 63304, 3651 Rue De Renard, Florissant, Missouri 63034, 700 Rugby 
Court, St. Louis, Missouri 63141, 3929 Justice Road, Florissant, Missouri 63034, and 
1182 Greenman Drive, Fenton, Missouri 63026 have invented new and useful 
improvements in 
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CROSS-REFERENCE TO RELATED APPLICATIONS 

This application is a continuation-in-part application, based upon co-pending 
U.S. Patent Application Serial No. 09/356,129, filed on July 17, 1999, from which 
priority is claimed. 

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR 
DEVELOPMENT 

Not applicable. 
BACKGROUND OF THE INVENTION 

This invention relates to video recording, and more particularly, to a digital video 
recording system for use at Point of Sale (POS) locations, Automated Teller Machines 
(ATM's), and other locations where the monitoring of activities is important. 

Video recording equipment is in common use throughout industry and 
commerce. Conventional video recording systems are analog systems employing a 
video recorder which records images on a magnetic tape. The recorder used is a slow 
scan recorder whose video input is provided by a camera located at the POS or ATM. 
Recording is typically continuous over a predetermined period which is typically 24 
hours. If nothing of significance occurs during the recording period, the tape is simply 
rerun again through the recorder with the prior recorded images being recorded over. 

This process has a number of problems associated with it. In most installations, 
most of the time, nothing ever happens. Rather, the same tape is used over and over 
with the same background image being recorded over and over. Only if something of 
significance happens is the tape removed and replaced. Preferably, each tape should 
be removed, degaussed, and reinserted in the recorder at the end of each recording 
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interval. Or, each tape is changed every day with the tape removed being degaussed 
before its next use. However, this is seldom done because of the expense or 
impracticality of doing so. 

Because the same images are continuously being recorded, over time, the 
magnetic particles in the tape tend to align themselves in a particular orientation 
consistent with those images. This residual magnetization degrades the resolution 
capability of the tape and resulting quality of the recorded images. One problem law 
enforcement officers or security personnel have with the recording systems 
described above is that the poor quality of the recorded images makes it difficult to 
discern features or details of the events captured on the tape, including the 
identification of persons. 

Another problem with conventional recording systems is that simply because an 
event of significance has occurred, recording does not stop. Thus, in order to view the 
event, it is necessary to rewind the tape and scan through it to the place where the 
event is recorded. In situations where time may be critical, the resulting delay is very 
undesirable. 

U.S. Patent No. 5,996,023 to Winter et al. teaches the use of circular pre- 
alarm buffering at high frame rates. The circular pre-alarm buffer is used to delay a 
signal until a set time has passed and a decision can be made as to the occurrence 
of an activity of interest. The circular pre-alarm buffer is continuously overwritten 
unless an alarm input triggers the contents to be marked for permanent storage. In 
this manner, high frame rate recordings (15 fps or more) can be made during the 
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time of interest. The '023 Winter et al patent specifically rejects the use of 
continuous recording at low frame rates (e.g. 1 fps) as outlined as possibly missing 
important data. The teachings of the '023 Winter et a/, patent also discuss the 
detection of motion. However, there is no indication of how the motion detection is 
performed and specifically, the motion is not used to aid in the size reduction of the 
stored data. It is used to provide event related information for storage with the image. 

United States Patent No. US 5,371,551 to Logan et al teaches the use of a 
buffer to provide a time delay in viewing stored images with pause, reverse, and 
thumbnail functions. The '551 Logan et al. patent also teaches that read-out rates 
may be different than input rates to accommodate a limited storage capacity. The 
'551 Logan et a/, system cannot be readily adapted to systems with a large or 
unlimited storage capacity configured for continuous activity recording. 

United States Patent No. 6,101,276 to Adiletta teaches the detection of 
motion through comparison with a reference image, using macro blocks to determine 
two types of macro block coding, spatial and temporal. Spatial blocks are coded by 
encoding the raw block itself. Temporal blocks are coded by finding motion vectors 
relative to a reference block and then encoding the error between the reference 
block and the temporal block. The blocks are encoded by comparing them to 
reference images and assigning a code to each block. The complexity of the 
identification of motion vectors is processor intensive, and the use of these 
operations requires specialized hardware, with some of the processing being done in 
software. 
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United States Patent No. 6, 125,145 to Koyanagi also teaches the use of 
motion detection using a complex algorithm for brightness and color information. The 
'145 Koyanagi patent further teaches the use of motion range detection at various 
distances from the camera using a plurality of ranges. The '145 Koyanagi patent 
uses of one of three operator selected methods for updating the reference image, 
unconditional, conditional, or none. Reliance upon an operator selected method for 
updating a reference image introduces a source of error into the system, as the 
environmental conditions in an image field may change such that the update method 
originally selected by the operator is no longer the best method. 

United States Patent No. 5,731,832 to Ng teaches the generation of a 
difference matrix of average differences between reference-frame macro blocks and 
current-frame macro blocks to detect motion. This may be used for intrusion 
detection. As an ancillary function, the current image is recorded if the difference 
matrix profile exceeds a threshold. Of note is that the entire image is recorded rather 
than only the difference portions as in the present invention. Also, only the image 
containing the detected motion is recorded and there is no continuous recording of 
image data, which may cause the loss of important context information contained in 
either previous of subsequent image frames. 

Finally, United States Patent No. 5,229,850 to Toyoshima teaches the use of an 
external sensor to detect the occurrence of an event to be monitored and to trigger the 
recording of an image associated with the event, adding to the complexity of the 
system. 
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BRIEF SUMMARY OF THE INVENTION 

Among the several objects of the present invention may be a digital video 
recording (DVR) system and apparatus for use in recording transactions and other 
occurrences sufficiently important that it is desirable to record them. The DVR system 
replaces conventional analog tape recording systems now in use with a personal 
computer (PC) configured to process and store digital images. 

A further object of the invention is to provide a system which captures and 
stores high resolution images, and maintains the quality of the stored image regardless 
of the amount of time elapsed from when the image was recorded, the number times 
which the image is viewed, and how often the media on which the image is stored is 
overwritten. 

A further object of the invention allows prior recorded images to be played back 
repeatedly; even while the system is recording current images from a camera, and in 
which the playback speed of a recorded image can differ from the speed at which 
image were recorded. 

A further object of the invention is a system in which recorded images are date 
and time stamped for random access search and recall rather than by a linear 
rewind/fast forward search and recall such as is done with conventional recording 
systems. Viewing of a stored image is done either locally or at a remote location. If 
done remotely, the image is transmissible over standard telephone lines without a 
degradation in image quality. 
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A further object of the present invention is a DVR system in which recorded 
images are compressed so to greatly increase the storage capability of the system. 
Image compression ratios are not constant, but rather are determined by various 
system factors. A compression algorithm is selected for compressing the image as a 
function of image content, location of the image within a scene being recorded, and the 
quality, resolution, and threshold of the compressed image, etc. 

A further object of the present invention is a DVR system in which images are 
authenticated for future usage as evidence of a crime, for example. Authentication 
prevents tampering with the recorded material. 

A further object of the present invention is a DVR system which is readily 
expandable such that the processor can receive inputs from multiple cameras without 
degrading the image processing capacity. The storage capacity of the DVR system 
may be readily expanded, enabling the system to provide continuous coverage for 
multiple cites with a separate camera being located at each site, but with image 
processing being accomplished at a remote location so monitoring at any particular site 
is not compromised. 

A further object of the present invention is a DVR system which purposefully 
records images at a lower frame rate to allow for continuous recording without regard 
to any particular activity. In this manner, all motion of interest in a camera field-of- 
view may be examined at a later time rather than just during a limited time interval. 
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A further object of the present invention is a DVR system is the use of detected 
changes in the scene to record only those portions of an image which have changed 
relative to a reference image, thereby reducing the storage of redundant or 
unimportant data. The added compression of the current invention due to the storing 
only of changed image areas precludes the need for buffering. Any area of the 
recorded data can be read out simultaneously with writing new data. For example, 
images which are 30 days old may be read out just as easily as those that are a few 
seconds old. This type of access cannot be performed using buffering techniques. 
The current invention does not require the use of buffers or time delays since all 
activity is recorded in a continuous manner. 

A further object of the present invention is a DVR system is the simplification of 
the recording process to preclude the use of hardware specific encoders. 
Specifically, motion vectors are not sought since determination of these vectors is 
very processor intensive and requires specialized hardware to allow real time 
performance. This situation is aggravated when attempting to record data from 
multiple camera inputs. 

A further object of the present invention is a DVR system in which motion is 
specifically excluded as an aid in the encoder. The current invention relies on the 
simpler concept of change in an image area, wherein any change in a specified area 
results in the storing of that area for the current image. Any area or region in an 
image which is not changed from a reference image is not stored. This simplification 
dramatically reduces processor requirements and the amount of data which must be 
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stored. No motion vector searches must be performed and no image data is stored 
unless changed from a reference. No distinction between spatial and temporal blocks 
is required and no coding of blocks is performed. Rather, each block with changes 
detected is simply encoded using a standard image compression algorithm such as 
JPEG. 

A further object of the present invention is a DVR system in which only intensity 
information is used in the identification of changed image areas, allowing for fast 
identification of the image areas to be compressed and stored. This allows for 
continuous recording of images at low data rates which results in many days of 
storage being available. 

A further object of the present invention is a DVR system maintains a simple 
algorithmic approach to determine when to update the reference image. The update 
method of the current invention automatically accounts for memory requirements, 
time between references, and image change considerations when making a 
determination if a new reference is needed. There is no reliance on an operator 
selection of update method to account for changed image environments. 

A further object of the present invention is a DVR system which does not rely 
on the computation of average values of macro blocks but rather on the comparison 
of individual pixels within macro blocks to detect when an image change occurs 
between a reference macro block and a current image macro block. Detection or 
identification of motion is of no significance, nor is the identification of motion when 
considering the scene as a whole. Specifically, the DVR system of the present 



Express Mail No: EL 820487646 US 



9 



invention does not exclude light changes or other variations which may result in a 
similar average value between a current macro block and a reference macro block. 
The various objects of the invention simplify the change detection and allow for the 
compression and storage of only the changed macro blocks. 

A further object of the present invention is a DVR system which uses only image 
information, time, and amount of data stored to determine when to take a new 
reference image. 

A further object of the present invention is a DVR system in which no external 
sensors are utilized to determine when to capture a new reference image. 

In accordance with the present invention, generally stated, a digital video 
recording system is for use at ATM machines, POS locations and the like. A video 
camera is directed at a scene of interest. The camera continuously views the scene 
and generates video images of the scene. An image processor evaluates digital 
signals representative of the video images, and in doing so compares the video image 
represented by a digital signal with a previously established reference image of the 
scene to determine if any changes have occurred therein. The image processor 
includes a memory in which a compressed video image is stored together with time 
and date information as to when the image was acquired. A video playback capability 
allows the memory location where an image of interest is stored to be directly 
accessed to retrieve the image. The playback system enables the memory associated 
with any desired date and time location to be directly accessed so an image of interest 
can be reviewed without having to sequentially scan video images in a date/time 
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sequence. Further, accessing the memory is done without interrupting the image 
processor's ongoing evaluation of video images. The system may be adapted for use 
with multiple cameras. Other objects and features will be in part apparent and in part 
pointed out hereinafter. 

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS 

In the drawings, Figure 1 is a representation of an installation with which the 

DVR system of the present invention is used; 

Figure 2 is a simplified block diagram of a prior art recording system; 

Figure 3 is a block diagram of a first embodiment of the DVR system of the 

present invention; 

Figure 4 illustrates the frame rate at which video images are acquired and 
processed and at which they are retrieved and played back; 

Figure 5 is a block diagram of a second embodiment of the system; and, 

Figure 6A and 6B represent a reference image obtained by the camera (Figure 
6A), and an image (Figure 6B) processed by the system to determine if there are any 
differences with respect to the reference. 

Corresponding reference characters indicate corresponding parts throughout 
the drawings. 

DETAILED DESCRIPTION OF THE INVENTION 

Referring to the drawings, a conventional monitoring system S utilizes a camera 
C which views a scene of interest. In security systems, this could be the interior of a 
bank, an Automatic Teller Machine (ATM), the counter of a fast food store, etc. Video 
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images produced by the camera are transmitted to recorder R which can be on the 
premises or remotely located. The images are recorded on a magnetic tape M or the 
like. When the tape is full, it is rewound and recorded over; or it is stored away and 
replaced with a new tape. If something of consequence occurs, in order to view it, the 
tape is transferred to a playback unit P where it is rewound to the beginning. The tape 
is then scanned to the point where recordation of the occurrence commences for 
viewing by interested parties. As noted, if the tape is simply recorded on over and over, 
the recorded image is degraded and vital information may be lost or rendered 
incomprehensible. 

As shown in Figure 3, a digital video recording system of the present invention 
is indicated generally 10. The system first includes a camera 12 which can be a 
conventional video camera. An example of the use of camera 12 is shown in Figure 1. 
An ATM machine A comprises an enclosure E. A monitor T, slot L for insertion of a 
customers ATM card, and pushbuttons B allow the customer to enter transactional 
information into a cash dispensing system enclosed in the machine. Once a 
transaction is authorized, cash is dispensed to the customer through a second slot L\ 
Camera 12 is mounted in the machine above the monitor at a level that allows the 
camera to obtain a face-on view of the customer. A one-way mirror W is placed in front 
of the camera. Camera 12 generates video images F at a variable frame rate which is, 
as shown in Figure 4 for example, 15 frames/sec. The video image produced by the 
camera comprises an analog signal which is provided as an output from the camera. It 
will understood by those skilled in the art that while camera 12 typically operates in the 



Express Mail No: EL 820487646 US 



12 




visible range of the light spectrum, it does not have to. Camera 12 could, for example, 
be an infrared camera. 

The analog signal produced by camera 12 is transmitted from the ATM machine 
to an image processor 14 which is remotely located with respect to the machine. A 
frame grabber 16 is interposed between the camera and image processor. This unit 
performs two functions. First, as its name suggests, it selects (grabs) frames of video 
from the stream of video transmitted by the camera. As shown in Figure 4, frame 
grabber 16 is programmed to select every frame of video from the camera. In 
Figure 4, N = 3; however, N can be any convenient number. In addition, the frame 
grabber converts the video signal from an analog signal to a digital signal Vd capable 
of being processed by image processor 14. 

It will be evident to those skilled in the art that portions of the frame grabber 16 
may be modified if the camera 12 directly provides digital output. In that case, the 
frame grabber may merely be used to capture the desired frames from the incoming 
stream of digital data. The function of converting the video signal from an analog signal 
to a digital signal will be unnecessary as the digital video signal Vd is provided directly. 

In a preferred embodiment, image processor 14 comprises two main elements: 
a microprocessor 18, and a memory 20. The image processor 14 is effected, for 
example, using a personal computer (PC) as a platform, and 20 gigabytes of disk 
memory. Use of a PC enables image processing to be accomplished using any 
appropriately equipped and programmed computer, whether a desktop, laptop, or 
other. This greatly enhances the flexibility of the system. The large capacity of memory 
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20 enables between 50 and 100 days of video imagery to be stored from a single 
camera 12. Further, unlike conventional video tapes which have a recording (storage) 
capacity of a but few hours at most, and which is non-expandable, memory 20 is 
readily expandable. 

As represented by Figure 6A, an image obtained by camera 12 is established 
as a reference against which the contents of subsequently acquired images are 
compared by the microprocessor 18. The image represented by Figure 6A has, for 
example, a resolution of 480 pixels by 640 pixels. Microprocessor 18 is configured to 
divide this image into, for example, 32 pixel by 32 pixel segments S for processing. As 
shown in Figure 6A, the image is divided into n segments S1, S2, ...Sn-1, Sn. The 
contents of each image segment may include all or part of objects O which objects are 
fixed in the scene. In processing the digital image, microprocessor 18 is configured to 
first compare segment S1 of the reference frame Fr with segment S1 of the frame Fa 
being analyzed. If the two frame segments compare with each other, the 
microprocessor 18 repeats the analysis with the next two corresponding frame 
segment. In the simplified representations of Figures 6A and 6B, the microprocessor 
18 will note the differences in segments S5, and in S7-S9 of the frame Fa being 
compared with the reference frame Fr. The processor discards the remaining frame 
segments in which there were no differences between the two frames, and stores the 
frame segments where differences appear in the memory 20. 

If they do differ, frame segment S1 is marked for later storage. A preferred 
approach for comparing segments S is to perform a pixel by pixel comparison such 
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that, if any corresponding pixels have an absolute difference in intensity which exceeds 
a threshold, a count is incremented. If the count of such pixels exceeds a second 
threshold, the segment is marked as changed. After all the segments S1 through Sn 
are so examined, the marked frame segments where differences appear are stored in 
memory 20. 

It will be apparent that other methods for determining the difference between 
two frame segments may be utilized without fundamentally altering the process of 
detecting and storing only those frame segments which have differences. For example, 
in images from non-visual detectors, a pseudo color representation of the scene may 
be made such that changes in color are appropriate to use to detect segment 
differences. Other algorithms may be employed which do not modify the basic object of 
the invention which is to detect changed macro blocks and only store those changed 
macro blocks and the corresponding reference images in memory 20 in order to 
increase the data storage compression efficiency. 

Image processor 14 periodically updates the reference image against which 
newly acquired frames Fa are compared. Because the system is typically used for 
security monitoring purposes, it is inefficient for the system to react to every change 
which may occur in the scene observed by the camera. For example, if the camera in 
the ATM machine of Figure 1 observes vehicles or passersby in the vicinity of the 
machine, it is not necessarily important that their passage be noted. These background 
changes are not important. What is important are changes in the area immediate the 
machine. 
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It is evident from the previous description that the amount of data recorded will 
be greatly reduced if the reference image Fr contains only background data such that 
if an image Fa contains the same objects 0, nothing will be stored. It is also necessary 
to account for system memory requirements and time elapsed since the last reference 
image when deciding when to update the reference image. Thus, a preferred algorithm 
for determining when to update the reference image Fr is as follows. 

Let N be the number of segments S in any image be N. Further, let M be the 
number of segments S saved from all Fa since the last reference frame. Then do the 
following: 

If (time since last reference frame > 15 minutes) or (M > 40 times N) then 

take a new reference frame, Fr 
else if (time since last reference frame > 10 minutes) or (M > 20 times N) then 
if the current frame Fa has not changed compared to the previous frame 
Fa by more than 5 segments then 
take a new reference frame, Fr 

end if 

end if. 

Thus, a new reference frame is taken a minimum of every 15 minutes or when 
the storage exceeds 40 times N. A new reference is also taken if the last two frames 
Fa are nearly identical and the time since the last reference frame Fr is at least 10 
minutes or the storage is at least 20 times N. Typically, a new reference will be taken 
whenever there is little change between two images. This corresponds to the condition 
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that only background objects O are in the reference image. The specific times selected 
and the amount of storage selected are dependent on the characteristics of processor 
14 and so will change depending on the specific implementation. 

The periodic updating of the reference image in the manner described will 
minimize the amount of storage required for long periods of inactivity in the scene 
which camera 12 is monitoring. Only when a change in image is detected will the 
corresponding segments S be recorded. However, in some situations, the camera view 
may include areas which are not of primary interest such as background traffic in a 
store in which the enclosure E is situated. In some installations it will be desired by the 
users of the system to record this activity and in other situations the users of the 
system will be more interested in maximizing the storage and not recording the extra 
information. The background traffic may be of interest if the user wishes to determine 
what activity occurred immediately before an incident. For example, if the enclosure E 
is vandalized it would be advantageous to maintain the recording of activity preceding 
the time period of the damage. In other situations, there may be auxiliary cameras 
which can adequately provide these details such that the only desired recording for 
camera 12 is that of people which approach the enclosure E. Accordingly, the system 
may optionally incorporate a presence sensor 22 which signals the processor when, 
for example, someone approaches the machine. This signal may be used by the 
processor to only record segments S when the sensor indicates someone in close 
proximity to the monitor T. 
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Optionally, the system 10 may incorporate a presence sensor 22 which signals 
the processor when, for example, someone approaches the machine. At that time, the 
image processor 14 responds by refreshing the reference image Fr, and comparing 
subsequence image frames with the refreshed image Fr. 

In storing the digital video images in memory 20, image processor 14 performs 
a number of functions. To conserve space in memory 20, the video image is exercised 
using a compression algorithm which minimizes the amount of information required to 
be stored, while at the same time insuring that none of the information contained in the 
image is lost so a high resolution image is always available for viewing. The 
compression algorithm is used in conjunction with certain control rules which establish 
the format for storing information in memory 20, and retrieving it from the memory. For 
different applications, different compression algorithms are used and the particular 
algorithm for the attendant application is established by the control laws portion of the 
processor. 

Further, all stored video images are date and time stamped with the time 
of day and the date on which the image was recorded. This information is encoded as 
digital data appended to the other digital data representing the image which is stored in 
the memory with the image contents. In addition the images can also be stamped with 
a transaction identification (ID) obtained from an associated device. For example, a 
transaction ID produced when a user performs a specific action such as inserting an 
ATM card in slot L may be produced. A search of the recorded video can then be 
directed to the location where the ID is stored. 
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Optionally, an authentication algorithm is also exercised on the data stored in 
the memory. Authentication of video images is described in U.S. Patent No. 5,870,471, 
herein incorporated by reference, which is assigned to the same assignee as the 
present application. Authentication is important for evidentiary purposes in that it 
prevents someone from tampering with the contents of an image without the tampering 
being readily detected. The authentication algorithm not only provides an immediate 
indication of tampering, but also provides information as to which part of the image has 
been tampered with. It will be understood that not every stored image need be 
authenticated. 

After a video image has been processed and its contents stored, the image 
remains in memory 20 until it is recalled for playback. Since the memory has a large 
capacity, the image is stored for a substantial period of time; for example, 50-100 days. 
When memory 20 is full, image processor 14 will begin to overwrite the images 
previously stored, beginning with that image which was first stored. This "first-in, first- 
out" procedure insures that the more recent images are retained while those which are 
probably unimportant (since they have not been recalled for viewing during the 50-100 
day interval) are deleted. It will be noted that this is in sharp contrast to current video 
recording systems in which tapes are constantly being reused so that if the importance 
of a recorded image is not immediately recognized, it has a good chance of being 
permanently lost. 

System 10 further includes a digital playback capability which is indicated 
generally 30 in Figure 3. The playback portion of the system includes a processor 32 
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which again is based on a PC platform. Importantly, the playback portion of the DVR 
system can be remote from the image processing portion of the system. Accordingly, a 
modem 34, 36 is located with the respective processors for transmitting commands 
and data back and forth between them. Transmission is via a standard 
telecommunications line and image date is transferred at a rate of 56.6 kbps, for 
example. Processor 32 accesses disk memory 20 through the image processor 14 via 
the modem link. Digital images are then transmitted from the memory via the modem 
link to processor 32. This processor has an associated display 34 (a television monitor 
or computer screen, for example) on which the images are viewed. Since the stored 
images are compressed images, the images received by processor 32 are first 
decompressed before being viewed. For this purpose, the digital images are first 
operated on by a decompression algorithm. As shown in Figure 3, the decompression 
algorithm is also subject to certain control laws which determine which algorithm of a 
plurality of algorithms are used. After being decompressed by the appropriate 
algorithm, the reconstituted image is displayed, the image being the high resolution 
image presenting substantial detail to the viewer. 

The above described invention has several important features over 
conventional recording systems. First, microprocessor 18 and memory 20 can be 
remote from the location at which the camera is installed. In addition, the processor 32 
and display 34 can be remote from microprocessor 18 and memory 20. Second, the 
rate at which stored frames of video are acquired from the memory and displayed can 
be a substantially different than those at which the video images are processed. Third, 
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the playback feature of the system is used simultaneously with the image processing 
and storage feature. Fourth, stored images are time and date stamped so the 
processor in the playback portion can go directly to the location in the memory where 
images are stored to retrieve them so there is no need to sequentially scan the 
memory. Fifth, stored images retrieved from memory 20 can be downloaded to other 
media with the superior visual qualities of the digital images being maintained. 

Because the images are recorded in a digital format, off-loading or downloading 
images from memory 20 to another memory, regardless of the media in which that is 
done (diskette, tape, etc.), is accomplished without the loss of quality of the image. 
Generating multiple copies of the recorded image can also be done without affecting 
the quality of the image. Not only can the downloading be done to different media, but 
the rate at which this is done is variable from medium to another, and this can also be 
done while the system continues to process new images and store them in the 
memory. 

Finally, system 10 includes a security system 40 preventing unauthorized 
persons from accessing the system. System 40 includes, for example, a password 
security feature which prevents one access to the system unless they possess the 
appropriate password. As is well-known in the art, such a system requires the user to, 
for example, type their password into a comparator which matches the entry against 
authorized passwords and allows the user access only if there is a match. 

Referring now to Figure 5, an alternate embodiment of the system is indicated 
generally 60. System 60 includes a plurality of cameras 12a-12n, each observing 
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different scenes or different aspects of the same scene. Each camera 12a-12n has an 
associated frame grabber 16a-16n for converting incoming video signals Va-Vn to 
digital signals Vda-Vdn. It will be appreciated that different camera/frame grabber pairs 
may operate at different frame rates and that an image processor 64 of the system 
accommodates all the frame rates for processing purposes. 

It will be evident to those skilled in the art that portions of the frame grabbers 
16a-16n may be modified if the cameras 12a-12n directly provide digital output. In that 
case, the frame grabbers may merely be used to capture the desired frames from the 
incoming stream of digital data, or may be eliminated completely. The function of 
converting the video signal from an analog signal to a digital signal will be unnecessary 
as the digital video signals Vda - Vdn are provided directly. 

Image processor 64 includes a PC based microprocessor 68 and an associated 
memory 70. The microprocessor 68 processes the digital signals Vda through Vdn and 
compares the video image represented thereby with a reference frame Vra through 
Vrn for the corresponding scene being observed. 

After processing, each digital signal is stored in memory 70 after first being 
compressed using an appropriate compression algorithm. It will be understood that 
different compression algorithm may be used for all the images stored in the memory; 
or that a different compression algorithm may be used for each image from a different 
observed scene. 

In memory 70, the images from the cameras are assigned their own memory 
space. Thus, as shown in Fig. 5, all images from camera 12a are stored contiguously 
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in one section of memory, all images from camera 12b are stored contiguously in 
another section of memory, and so forth with each cameras images having its own 
storage area. In this way, the cameras can have different frame capture rates and can 
have different amounts of storage allocated to the camera. This allows for different 
amounts of storage for each camera. For example, in the case of a two camera system 
with cameras 12a and 12b, 60% of the storage could be allocated to camera 12a and 
40% of the storage could be allocated to camera 12b. Assuming the cameras are 
viewing similar scenes, assigning the same frame capture rate and compression 
technique to both cameras will result in the ability to store 50% more data for camera 
12a than for camera 12b. Alternatively, if the frame capture rate for camera 12a is 
made 1.5 times as great as that for camera 12b then the allocated storage area will 
accommodate the same number of days of storage for both cameras. 

The separate storage of images from each camera also facilitates the retrieval 
of image data by playback system 30. An operator may request the data from an 
individual camera and it is readily available from its assigned storage area. 

Those of ordinary skill in the art will recognize that alternative storage methods 
for multiple video images are possible. For example, in the memory 70, the images 
from the cameras may be interleaved. In interleaved storage, an image from camera 
12a is stored in the memory, then an image from camera 12b, and so forth until an 
image from camera 12n. The storage cycle is then repeated. Again, it will be 
understood that this order is exemplary only and that the order in which the images are 
stored is a function of the actual sequence in which the images are processed. If a 
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particular camera produces images at a frame rate significantly greater than that of 
another camera, images from the one camera will be stored more frequently than 
those from the other. Or, if one camera is shut down while others remain operating, 
only the frames from the operating camera are interleaved. 

Another feature of the recording system 60 is the ability to manipulate the input 
images to form a multiplex of several images for storage. In this manner, if it is desired 
to see what happens in several views simultaneously, only a single image need to be 
stored, retrieved, and viewed. For example, the images Vda, Vdb, Vdc, and Vdd 
produced from cameras 12a, 12b, 12c, and 12d may all be reduced to one quarter 
their normal size. These one quarter size images may then be combined into a single 
full size image by placing Vda in the upper left comer, Vdb in the upper right corner, 
Vdc in the lower left comer, and Vdd in the lower right corner. The normal compression 
and storage of this composite image may then be performed. A separate storage area 
in memory 70 may be assigned for the composite image. In addition, if desired, the 
original full size images may be stored in their own assigned storage areas. It will be 
understood that many such combinations may be made and are limited only by the 
controls made available to the operator of the system. 

As with system 10, system 60 includes security system 40 to prevent 
unauthorized persons from accessing the system. 

Although not shown in Figure 5, system 60 includes a playback capability 
similar to that of system 10. Operation of the playback portion of the system is similar 
to that previously described. It will appreciated that, as with the playback portion of 
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system 10, microprocessor 68 and memory 70 are remote from the locations where 
playback occurs. Again, the rates at which stored frames of video are obtained from 
memory 70 are different than the rates at which the video images are processed by 
microprocessor 68 and images are played back simultaneously with newly acquired 
images being processed and stored. Since the stored images are time and date 
stamped, the processor of the playback portion of the system can go directly to their 
storage location in the memory to retrieve the images. 

Any images to be replayed are either reference images, which may be 
retrieved, decompressed, and displayed directly, or changed image blocks, which 
must be combined with a reference image to produce an image at the time 
requested. The combining of changed image blocks with the reference image blocks 
is performed by decompressing the reference image, decompressing the changed 
image blocks from the subsequent image, and then replacing the corresponding 
image blocks in the reference image with the changed image blocks from the 
subsequent image. In order to determine which blocks in the reference image must 
be replaced, an associated block map is stored for each group of changed image 
blocks which determines their location in the reference image. A preferred map is to 
keep a binary representation of all the blocks in the image such that a single bit in 
the memory map determines if each block is changed or unchanged. For example, a 
0 in the map may indicate an unchanged block and a 1 a changed block. Thus, there 
are as many bits in the image map as there are blocks in each digital image. The 
image map bits may be stored in order to correspond to a position in the image. For 
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example, starting at the upper left corner of the image and provide the first bit in the 
image map as corresponding to this location. Subsequent bits may then correspond 
in order to the blocks proceeding from the upper left corner to the upper right corner 
and then from the left side of the second row to the right side of the second row and 
so forth until the entire image is accounted for. For a 640 by 480 pixel image divided 
into blocks of 32 pixels by 32 pixels, this will result in 20 blocks across the image and 
20 bits in the image map corresponding to the top row of blocks. There are then 15 
rows of blocks and consequently a total of 20 times 15 or 300 bits required in the 
image map. The resulting image map so described may be stored directly or may be 
compressed to conserve storage space in memory. 

As any individual image may be recreated by retrieving the changed macro 
blocks and the corresponding reference image, it is evident that this process 
facilitates all the normal replay functions such as forward play, reverse play, fast 
forward, fast reverse, pause, etc. In addition, using the transaction ID associated with 
corresponding images it is evident that a transaction ID may be found and the 
corresponding image displayed without having to replay any other images. 

Finally, the stored images can be downloaded from the memory 70 to other 
storage media, such as tapes, CD-ROMs, or other magnetic media without loss in the 
quality of the image. 

What has been described is a DVR system for recording financial transactions 
or other important occurrences. The system employs a video camera and an image 
processor having a memory capability. The system captures, processes and stores 
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video images, and maintains the high resolution (quality) of the stored image 
regardless of how much times lapses before the image is viewed, and how often the 
image is viewed. Review of a stored image can be done while the system continues to 
acquire and process new images. The use of image compression greatly increases the 
memory storage capability of the system, and image compression ratios are not 
constant but vary as a function of the aspect of the object viewed and other factors. 
Recorded images are time and date stamped and authenticated for future usage. The 
system is expandable for use with multiple cameras and processors and with an 
expandable memory. 

In view of the foregoing, it will be seen that the several objects of the invention 
are achieved and other advantageous results are obtained. As various changes could 
be made in the above constructions without departing from the scope of the invention, 
it is intended that all matter contained in the above description or shown in the 
accompanying drawings shall be interpreted as illustrative and not in a limiting sense. 
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